Add support for network data in Config Drive by vishesh92 · Pull Request #9329 · apache/cloudstack

vishesh92 · 2024-07-03T07:41:23Z

Description

This PR fixes #2872
PR adds support for Dhcp & Dns capabilities for ConfigDrive network element.

Docs PR: apache/cloudstack-documentation#412

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Create a network offering with DHCP, DNS & UserData provider with ConfigDrive
Create a network with the above offering
Launch an instance using a cloud init configured template.
Instance gets the assigned IP, DNS & Userdata.

Note

While attaching secondary nics, interface is assigned an IP on boot only if the instance was started for the first time.
Updating of IPs & NICs after first boot doesn't reflect on the instance even after reboot.

How did you try to break this feature and the system with this change?

codecov · 2024-07-03T07:49:13Z

Codecov Report

Attention: Patch coverage is 57.27700% with 91 lines in your changes missing coverage. Please review.

Project coverage is 15.71%. Comparing base (47a6b70) to head (6c26bc4).
Report is 31 commits behind head on main.

Files	Patch %	Lines
...oud/network/element/ConfigDriveNetworkElement.java	21.42%	42 Missing and 2 partials ⚠️
...tack/engine/orchestration/NetworkOrchestrator.java	0.00%	21 Missing ⚠️
...dstack/storage/configdrive/ConfigDriveBuilder.java	83.33%	2 Missing and 16 partials ⚠️
.../main/java/com/cloud/network/NetworkModelImpl.java	0.00%	7 Missing ⚠️
...oudstack/storage/configdrive/ConfigDriveUtils.java	95.23%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #9329      +/-   ##
============================================
+ Coverage     15.53%   15.71%   +0.17%     
- Complexity    11988    12269     +281     
============================================
  Files          5496     5502       +6     
  Lines        481450   490878    +9428     
  Branches      60738    64541    +3803     
============================================
+ Hits          74809    77156    +2347     
- Misses       398364   405194    +6830     
- Partials       8277     8528     +251

Flag	Coverage Δ
uitests	`4.98% <ø> (+0.80%)`	⬆️
unittests	`16.45% <57.27%> (+0.14%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

vishesh92 · 2024-07-03T07:59:13Z

@blueorangutan package

blueorangutan · 2024-07-03T08:00:04Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-07-03T08:59:32Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 10238

blueorangutan · 2024-07-03T11:15:14Z

[SF] Trillian Build Failed (tid-10722)

blueorangutan · 2024-07-03T11:44:23Z

[SF] Trillian test result (tid-10723)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t10723-kvm-centos7.zip
Smoke tests completed. 0 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
login_test_saml_user	`Error`	3.05	test_login.py

vishesh92 · 2024-07-05T07:00:50Z

@blueorangutan package

blueorangutan · 2024-07-05T07:02:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-07-05T07:10:49Z

Packaging result [SF]: ✖️ el7 ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 10255

vishesh92 · 2024-07-05T09:50:26Z

@blueorangutan package

blueorangutan · 2024-07-05T09:52:04Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-07-05T11:01:57Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10261

harikrishna-patnala

few comments from my side @vishesh92 can you please address them

...hestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java

.../configdrive/src/main/java/org/apache/cloudstack/storage/configdrive/ConfigDriveBuilder.java

server/src/main/java/com/cloud/network/element/ConfigDriveNetworkElement.java

harikrishna-patnala

Another question, how this needs to be configured in the network offering ? like we choose the config drive as the provider for userdata while creating the network offering

.../configdrive/src/main/java/org/apache/cloudstack/storage/configdrive/ConfigDriveBuilder.java

vishesh92 · 2024-07-11T07:00:11Z

Another question, how this needs to be configured in the network offering ? like we choose the config drive as the provider for userdata while creating the network offering

Yes. You just need to select confidDrive for Dhcp & DNS while creating the network offering.

blueorangutan · 2024-08-16T07:56:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-08-16T09:08:39Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10681

blueorangutan · 2024-08-17T01:53:07Z

[SF] Trillian test result (tid-11091)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57487 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11091-kvm-ol8.zip
Smoke tests completed. 135 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_DeployVmAffinityGroup	`Error`	1.46	test_affinity_groups.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	`Error`	70.60	test_network.py
test_01_non_strict_host_anti_affinity	`Error`	3.61	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	`Error`	3.65	test_nonstrict_affinity_group.py
test_02_unsecure_vm_migration	`Error`	425.52	test_vm_life_cycle.py

blueorangutan · 2024-08-20T06:49:42Z

[SF] Trillian Build Failed (tid-11103)

blueorangutan · 2024-08-20T23:33:56Z

[SF] Trillian test result (tid-11106)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54415 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11106-kvm-ol8.zip
Smoke tests completed. 136 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_DeployVmAffinityGroup	`Error`	1.48	test_affinity_groups.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown	`Error`	212.25	test_network.py
test_01_non_strict_host_anti_affinity	`Error`	2.58	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	`Error`	3.58	test_nonstrict_affinity_group.py

vishesh92 · 2024-08-21T05:41:29Z

@blueorangutan package

blueorangutan · 2024-08-21T05:42:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-08-21T06:55:10Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10722

blueorangutan · 2024-08-22T00:18:46Z

[SF] Trillian test result (tid-11124)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 53325 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11124-kvm-ol8.zip
Smoke tests completed. 135 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_DeployVmAffinityGroup	`Error`	1.48	test_affinity_groups.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown	`Error`	192.56	test_network.py
test_01_non_strict_host_anti_affinity	`Error`	3.58	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	`Error`	3.58	test_nonstrict_affinity_group.py
test_01_secure_vm_migration	`Error`	135.40	test_vm_life_cycle.py
test_01_secure_vm_migration	`Error`	135.40	test_vm_life_cycle.py

vishesh92 · 2024-08-22T12:16:02Z

@blueorangutan package

blueorangutan · 2024-08-22T12:18:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-08-22T13:24:42Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10753

DaanHoogland · 2024-08-22T13:57:31Z

@vishesh92 does this need more testing?

DaanHoogland · 2024-08-22T13:57:45Z

@blueorangutan test

blueorangutan · 2024-08-22T13:58:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-08-23T07:00:16Z

[SF] Trillian test result (tid-11145)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 58696 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11145-kvm-ol8.zip
Smoke tests completed. 137 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	`Error`	2.30	test_network.py
test_01_secure_vm_migration	`Error`	404.46	test_vm_life_cycle.py
test_02_unsecure_vm_migration	`Error`	450.63	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	`Error`	359.84	test_vm_life_cycle.py

blueorangutan · 2024-08-23T21:00:57Z

[SF] Trillian test result (tid-11152)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 50517 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11152-kvm-ol8.zip
Smoke tests completed. 139 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2024-08-26T08:36:34Z

@vishesh92 , this had quite some changes since functional tests, does it need a re-visit, or can we merge now?

vishesh92 · 2024-08-26T08:49:56Z

@vishesh92 , this had quite some changes since functional tests, does it need a re-visit, or can we merge now?

We can merge now

weizhouapache · 2024-09-05T08:20:17Z

@vishesh92
there are lots of smoke test failures with test_01_deployVMInSharedNetwork, mostly happened on vmware environments.

worth checking , maybe related to the template ?

vishesh92 · 2024-09-05T08:29:11Z

@vishesh92 there are lots of smoke test failures with test_01_deployVMInSharedNetwork, mostly happened on vmware environments.

worth checking , maybe related to the template ?

@weizhouapache Let me check and create a fix pr for that.

boring-cyborg bot added component:networking component:orchestration component:storage labels Jul 3, 2024

vishesh92 force-pushed the configdrive-network-data branch from 227be3d to 9d59c87 Compare July 3, 2024 07:58

vishesh92 force-pushed the configdrive-network-data branch from 8ffa213 to 6439aa7 Compare July 5, 2024 09:50

vishesh92 force-pushed the configdrive-network-data branch 2 times, most recently from b1f0fda to b8a1a90 Compare July 5, 2024 09:51

vishesh92 changed the title ~~WIP: Add support for network data in Config Drive~~ Add support for network data in Config Drive Jul 9, 2024

yadvr mentioned this pull request Jul 9, 2024

Improve ConfigDrive to store network information #2872

Closed

vishesh92 force-pushed the configdrive-network-data branch from 73d2970 to e438a3f Compare July 9, 2024 13:17

vishesh92 mentioned this pull request Jul 10, 2024

Document network support for configdrive apache/cloudstack-documentation#412

Merged

harikrishna-patnala reviewed Jul 10, 2024

View reviewed changes

vishesh92 commented Jul 10, 2024

View reviewed changes

.../configdrive/src/main/java/org/apache/cloudstack/storage/configdrive/ConfigDriveBuilder.java Outdated Show resolved Hide resolved

boring-cyborg bot added the component:vpc label Jul 11, 2024

yadvr added this to the 4.20.0.0 milestone Jul 11, 2024

vishesh92 force-pushed the configdrive-network-data branch from ccf3568 to ee9507e Compare July 12, 2024 08:39

vishesh92 force-pushed the configdrive-network-data branch from 665cdff to 425af71 Compare August 21, 2024 05:41

Update test/integration/smoke/test_network.py

6c26bc4

vishesh92 force-pushed the configdrive-network-data branch from 425af71 to 6c26bc4 Compare August 22, 2024 12:15

DaanHoogland merged commit bc28665 into main Aug 26, 2024

DaanHoogland deleted the configdrive-network-data branch August 26, 2024 08:54

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Aug 30, 2024

Add support for network data in Config Drive (apache#9329)

7f26e6d

kiranchavala mentioned this pull request Sep 19, 2024

Support ConfigDrive option in VPC network offerings #9706

Closed

phsm mentioned this pull request Dec 12, 2024

enhancement: add password to configdrive vendor_data.json #10061

Merged

14 tasks

weizhouapache mentioned this pull request Dec 16, 2025

VR healthchecks fail on VPC with ConfigDrive (rather than VirtualRouter) #12265

Closed

Conversation

vishesh92 commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Note

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vishesh92 commented Jul 3, 2024

Uh oh!

blueorangutan commented Jul 3, 2024

Uh oh!

blueorangutan commented Jul 3, 2024

Uh oh!

blueorangutan commented Jul 3, 2024

Uh oh!

blueorangutan commented Jul 3, 2024

Uh oh!

vishesh92 commented Jul 5, 2024

Uh oh!

blueorangutan commented Jul 5, 2024

Uh oh!

blueorangutan commented Jul 5, 2024

Uh oh!

vishesh92 commented Jul 5, 2024

Uh oh!

blueorangutan commented Jul 5, 2024

Uh oh!

blueorangutan commented Jul 5, 2024

Uh oh!

harikrishna-patnala left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

harikrishna-patnala left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vishesh92 commented Jul 11, 2024

Uh oh!

blueorangutan commented Aug 16, 2024

Uh oh!

blueorangutan commented Aug 16, 2024

Uh oh!

blueorangutan commented Aug 17, 2024

Uh oh!

blueorangutan commented Aug 20, 2024

Uh oh!

blueorangutan commented Aug 20, 2024

Uh oh!

vishesh92 commented Aug 21, 2024

Uh oh!

blueorangutan commented Aug 21, 2024

Uh oh!

blueorangutan commented Aug 21, 2024

Uh oh!

blueorangutan commented Aug 22, 2024

Uh oh!

vishesh92 commented Aug 22, 2024

Uh oh!

blueorangutan commented Aug 22, 2024

Uh oh!

blueorangutan commented Aug 22, 2024

Uh oh!

DaanHoogland commented Aug 22, 2024

Uh oh!

DaanHoogland commented Aug 22, 2024

vishesh92 commented Jul 3, 2024 •

edited

Loading

codecov bot commented Jul 3, 2024 •

edited

Loading